Computer-Aided Language Processing

The invited talk will introduce Computer-Aided Language Processing (CALP) as an emerging area of practical interest and significance and will present experiments of several CALP applications which show that while no quality is lost, much time is saved when employing CALP tools. Computer-aided language processing: context, history and examples The past years have seen a variety of promising Natural Language Processing (NLP) projects but in the vast majority of real-world applications, fully automatic NLP is still far from delivering reliable results the simple reason being that computers are not as good as humans at understanding human languages. In fact, pre-processing tools are far from perfect. By way of example, the best accuracy reported in robust parsing of unrestricted texts is around the 90% mark (Charniak 2000) whereas best performing anaphora resolution programs score only in the 60%s (Mitkov and Hallett, forthcoming). As a result, computer-aided methods have emerged as a practical alternative. In the computer-aided scenario, processing is not done entirely by computers, but human intervention improves, post-edits or validates the output of the computer program. The invited talk will introduce the historical background of Computer-Aided Language Processing (CALP) by outlining the first example of CALP. In 1980 Martin Kay (Kay 1980) proposed the idea of Machine-Aided Translation as "A cooperative man-machine translation system, leaving the 'mechanical and routine' translation work to the computer and 'the more rewarding, more exciting' activities to the human translator". Machine Translation (MT) offers further (and more typical) examples of computer-aided language processing such as MT relying on post-editing. In addition, the popular Compilation copyright © 2006, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Translation Memory (TM) techniques can be regarded as another example. A number of applications based on CALP have been reported recently including these in the area of text summarisation (e.g., Orasan, Mitkov and Hasler 2003), automatic generation of multiple-choice tests (Mitkov and An, 2003; 2005) and information extraction (Cunningham et al. 2002). Annotation tools operating in semi-automatic mode and offering the user the opportunity to speed up the mark-up of corpora or data in general, can also be regarded as CALP applications (McEnery 2003). Further examples of applications/tasks where CALP has played an important role include acquisition of semantic lexicons and ontologies from text, semantic annotation and term extraction. Applications in focus and experiments The presentation will discuss the results of three CALP projects and their related experiments. The first one includes experiments in Machine-Aided Translation (MAT) and finds out that by using an MAT tool the translators are able to gain considerable time saving, translating 1.8 times faster with the help of the MAT tool (or otherwise said, the efficiency factor of this CALP task is assessed to be 1.8). The second experiment was conducted as part of the development of the computeraided summarisation tool CAST with the results suggesting an efficiency factor of 1.25 when using CAST. The third and most extensive evaluation study was part of a project on automatic generation of multiple-choice tests. A computer-aided tool was developed to assist test developers on the time-consuming and labour-intensive task of manual test construction. The tool was formally evaluated not only for its efficiency – whether time was gained when using the tool to construct multiple-choice tests, but also for its quality – whether the tests constructed with the help of the tool were as good as the ones manually produced. The evaluation concluded that the tests produced Compilation copyright © 2006, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.